Culture shapes how we describe facial expressions

From Darwin through Wittgenstein to contemporary scientific investigations, it has been argued humans tend to view facial expressions through a mentalistic lens. According to this view, when looking at someone’s expressive face, we see emotion and are unable to describe the face in behavioral terms, i.e., name the details of facial movements. At the same time, however, a growing body of work shows cultures in fact differ in the degree of importance they attribute to mental states and willingness to discuss them. Is this variation reflected in the linguistic coding of facial expressions? To explore this, we conducted two facial expression naming tasks targeting mental states and facial movements with speakers of Maniq (Austroasiatic, Thailand) and Polish (Indo-European, Poland), representing highly diverse linguistic and cultural settings. We found that, while Polish speakers conformed with the predicted orientation towards mental states, this did not hold for Maniq speakers. The Maniq were instead oriented towards behavioral aspects of faces, naming them more frequently, more precisely, and with higher consensus, compared to the Polish. These differences are carved into the Maniq and Polish lexicons, suggesting diverse cultures exhibit differential specialization in verbalizing expressive faces.


General description of the study populations 1.Maniq
The Maniq are a traditional hunter-gatherer group living in a tropical rainforest in Southern Thailand.The population numbers about 300 members, living in groups of an average size of 25-35 people.Most Maniq are nomadic, with some variation from group to group .The group composition and residential unit size vary across the year.The Maniq inhabit the area of the Banthad mountain range stretching across four southern provinces of Thailand (Trang, Satun, Phatthalung, and Songkhla).The group participating in this study is from the area of the Manang district, Satun province.The Maniq society is organized according to egalitarian social norms and practices such as extensive sharing and social taboos against status seeking behaviors .
Maniq is an unwritten language of the Austroasiatic language family, the Aslian branch.Its formal properties include a rich phonemic inventory, intricate derivational morphology, frequent argument ellipsis, and semantically dense verb lexicon with highly specific meanings .All Maniq speakers are also good or fluent speakers of Southern Thai, used in interactions with outsiders.Maniq is the primary language of everyday communication and the first language learnt by children.

Polish
There is an estimated number of 40 to 45 million speakers of Polish in the world, of which about 38 million in Poland itself .Most Poles live in large urban areas and like other large, industrialized nations are exposed to modern technology, globalized media, and popular culture.Participants in this study were inhabitants of Warsaw, the Polish capital, and were speakers of a standard variety of Polish.
Polish is a Slavic language of the Indo-European language family.Unlike Maniq, it has a long history of written texts and a strong prescriptivist tradition in orthography and grammar.
It contains borrowed vocabulary from other Slavic, Romance, and Germanic languages, including many lexical items which are cognates of modern English, French, and German words (such as the mental state terms ekscytacja 'excitement', satysfakcja 'satisfaction', irytacja 'irritation').About two thirds of adult Polish speakers (age 18-69) also speak a foreign language, most commonly English, with levels of proficiency varying from basic, intermediate, to advanced .

Main response identification
Similar to other cross-linguistic investigations of codability , coding was focused on individuating semantic heads of each response, rather than morpho-syntactically-defined categories.The procedure was the same for Maniq and Polish and was sensitive to the linguistic systems of both languages, considering their structural features and irregularities (see below).
Both studies involved free description, so participants could provide multiple responses.We took into account all responses provided, unless the participants explicitly signaled a correction.When someone indicated they were unable to describe a clip, e.g., by saying "I don't know", "nothing comes to mind", etc., we coded that as "NR" (no response).If a response was missing due to experimenter's error, this was coded as "NA" (nonapplicable).
In our coding, we identified main responses, i.e., the elements carrying the semantic gist of the descriptions, which constituted further units of analysis.To do that, we removed any function words, grammatical morphemes, and modifiers, and extracted the common roots of formally related responses.We also omitted body part names since the key information we were interested in for the two languages was how easy it was to name the actions and states of the persons in the videos in each of them.Similarly, we disregarded those parts of the responses which did not describe the person in the video but elaborated on imagined background scenarios.A few examples illustrating the basic coding principles are provided below: (1) Myślę, że jest to forma jakiejś aprobaty (Polish) 'I think it's a form of approval.'Main responses: 1. aprobata 'approval' (2) Ruch brwi, zmarszczenie brwi, no i tyle (Polish) 'Eyebrow movement, eyebrow wrinkling, and that's it.'Main responses: 1. ruch 'movement' 2. zmarszczenie 'wrinkling' (3) ʔɛʔ limiŋyeŋ, mɛt hay dɔn (Maniq) 'She's showing the whites of her eyes, her eyes are like that of a slow loris (Nycticebus coucang).'Main responses: 1. lŋyeŋ 'to show the whites of one's eyes' 2. hay dɔn 'like slow loris' (4) ʔɛʔ ʔiyay ɡanaʔ, wa pãʔ ʔisiʔ (Maniq) 'He's upset about his partner, she went off to another man.' Main responses: 1. ʔiyay 'to be upset' For responses involving complex predicates, where both verbs contributed lexical information, e.g., multi-verb constructions (Maniq) or converbal constructions (Polish), each verb from the predicate was included as a distinct response, e.g., in pimikluk cimintũn [5][6][7][8][9] 'smiling raising upper lip' (Maniq), both the verb pikluk 'to smile/laugh' and citũn 'to raise upper lip' were main responses.For modal complement-taking predicates, e.g., próbować ukryć 'try to hide something' (Polish), chcieć wydać dźwięk 'want to make a sound' (Polish), the main response was the entire predicate.
Formally related responses which shared a common root were coded as the same category, e.g., the Polish noun ekscytacja 'excitement' and adjective podekscytowany 'excited'.Similarly, morphologically complex verbs and nouns with shared roots and distinct prepositional prefixes were collapsed, but only if their meaning was highly similar, e.g., podnosić 'to raise' (under-carry) and u-nosić 'to raise' (at-carry), and not if there was a semantic difference between them, e.g., przy-mykanie 'closing slightly' and za-mykanie 'closing entirely'.If words sharing a common root differed in their semantics, however, we treated them as distinct.For example, in some cases two or more words were formally related, but represented only partially overlapping concepts, e.g., the Polish adjective zła 'angry/bad/evil' and noun złość 'anger'.Such cases were regarded as separate responses.

Response type classification
Once the main responses have been identified, we classified them according to response type.There were two main types of responses: "bodily actions" and "mental states".A small number of responses which did not fit easily into either of these two categories were classified as "other".
The bodily action category included descriptions of simple movements of facial muscles or the head, e.g., marszczyć brwi 'to wrinkle one's brows' (Polish), cikiey 'to turn one's head sideways or back' (Maniq), as well as descriptions of more complex physical actions, e.g., śmiać się 'to laugh' (Polish), kahey 'to cry' (Maniq), słuchać 'to listen' (Polish), and biyɔh 'to speak' (Maniq).Generally speaking, this category was associated with specific actions performed with the body, as opposed to more abstract actions (classified as "other"), e.g., udawać 'to pretend' (Polish), przyciągać uwagę 'to draw attention' (Polish), przywitać się 'to greet' (Polish), where bodily action was implied but the lexicalized meaning component concerned the actor's intentions and goals.
Finally, responses coded as "other" included a variety of descriptions not falling into either category.These included: abstract actions with lexicalized intention or goal, e.g., udawać 'to pretend' (Polish), physiological states and sensations, e.g., campɨs 'ill' (Maniq), perceived similarity to action or appearance of an animal, e.g., hay bəwac 'like a piɡ-tailed macaque' (Maniq), and general descriptions of appearance, e.g., suche usta 'dry lips' (Polish).Most of the responses classified as "other" were infrequent; in fact, the majority of them were used only a single time.The top three responses in this category were: NR '(no response)' (attested 26 times), nic 'nothing' (Polish) (attested 8 times), and potwierdzać 'to confirm' (Polish) (attested 6 times).

Ethnographic background to mental states and actions in the Maniq society
Observations from long-term ethnographic fieldwork with Maniq people suggest references to mental states are not a prominent part of everyday discourse.Though mental state terms such as ʔantɨŋ 'to be afraid' and məwɛ ŋ 'to feel lazy, reluctant' are common everyday words, it is rare for speakers to linger on descriptions of mental states or center conversations around them.Even in situations which seem particularly emotion-laden, people typically do not focus their narratives on mental states but tend to foreground external events.For instance, when in February 2014 one family's shelter caught fire while they were asleep and burnt down, members of this household related this story by highlighting key events and did not spontaneously reference mental states.
The low salience of mental states holds also in relation to other minds.Although the Maniq society does not seem to have an explicit opacity of mind doctrine similar to the one of Samoans , mental states of other people are typically not in focus in ordinary talk.In some contexts, e.g., when talking about strangers, it might be relevant to specify whether or not they are angry/aggressive (ɡəbaʔ), as threatening behavior by outsiders historically used to be, and sometimes still is, a concern for Maniq people .The importance of this determination is reflected in the presence of the phrase hɨc ɡəbaʔ 'not angry/aggressive' in the emotion naming task, with faces ranging from happy, neutral, to embarrassed, all receiving this description from time to time (note that no similar response was recorded in the Polish data).Beyond this coarse-level distinction, however, spontaneous references to mental states of others are relatively infrequent.When casually viewing old photos of their own group, for instance, Maniq speakers tend to reference actions and other physical details of the scene and only rarely mention any mental states.Thus, mental states do not tend to be focal in discourse, as reflected also in the Maniq lexicon for mental states being relatively small and semantically general.What cultural factors could explain this?Certainly, the Maniq society is not isolated in this phenomenon.
Lexical mental state distinctions are similarly coarse-grained in at least some of the close linguistic relatives of Maniq, e.g., Ceq Wong of Malaysia .It has been suggested talking openly about mental states of other people clashes with egalitarian values of personal autonomy, as it amounts to claiming undue authority over others, and might therefore be culturally constrained .This could well be a relevant factor for Maniq people, who as egalitarian hunter-gatherers attach importance to respecting the bounds of personal autonomy.However, the low salience of mental states is a broader phenomenon in the Maniq society, not limited exclusively to other minds and is best viewed in the wider context of emotional events and the Maniq concept of a person.
Rich anthropological research shows that the way persons are conceptualized is not a universal, but differs across societies and is fundamentally important for how people might conceive of their mental life .Crucially, the Maniq concept of a person has been described as "action-oriented", meaning that Maniq people define personhood primarily on the basis of actions rather than objectified essence (see also ref. ).For example, identifying someone as Maniq is critically dependent not on ethnicity, but on actions considered intrinsic to "Maniq-ness" such as hunting, gathering, and eating forest foods .
So, being able to call oneself Maniq necessarily requires engaging in these actions.In the context of emotion, similarly, considerable focus is placed on actions, i.e., what one does in the context of emotion-laden events.Private thoughts and feelings, on the other hand, are not foregrounded in the same way as they are in the Euro-American context .For instance, when asked about their reaction to the loss of a loved one, e.g., a mother, Maniq people typically mention what they do, e.g., visit and spend time with their siblings, and do not volunteer information about internal mental states.
These observations are consistent with the linguistic data presented in the article.In the emotion task, Maniq speakers used action descriptions half of the time, even though the instruction targeted mental states, whereas Polish speakers used primarily dedicated emotion terminology.In addition, the Maniq mental state lexicon itself is oriented to actions and is distinctly "verby" in its grammatical properties .While typical English translations of Maniq terms involve adjectives or nouns, e.g., 'to be angry', 'to feel anger', in Maniq by default they function as verbs and reveal a conceptualization akin to actions.Polish terms, on the other hand, are more flexible and depending on the specific concept can be coded as verbs, but they are also commonly expressed as nouns (unattested with mental state terns in Maniq).

Table S2 Information about the stimuli -emotion task Order of presentation Stimulus id Target emotion File names from Amsterdam Dynamic Facial Expression Set
Note:The letter F in the file name stands for 'female' and M for 'male' (actor).

Maniq: Domain-specific terms for facial movements A
list of all unique Maniq responses dedicated to facial movements used across the two tasks list of all unique Maniq responses categorized as mental states used across the two tasks list of all unique Polish responses categorized as mental states used across the two tasks A A